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IX  A  AS  TRACT  (Mjjdmur?. lOQwordti  During  the  past  year  these  investigators:  1)  Illustrated 
application  of  computer  science  to  neuroscience  at  three  levels:  measuring,  modeling, 
and  understanding  the  computational  function  of  the  colummar  pattern  of  ocular  domin¬ 
ance  in  primate  visual  cortex;  2)  Demonstrated  an  algorithm  for  modeling  polymap 
architecture  of  the  cerebral  neocortex,  where  the  term  "polymap"  emphasizes  the  join 
occurrence  of  topographic  mapping  of  multiple  sub-modalities,  interlaced  in  the  form 
of  macroscopic  patches  ("columns")  into  a  single  cortical  lamina;  3)  Considered  a 
space-variant  sensor  design  based  on  the  conformal  mapping  of  the  halp  dish,  w=log 
(z+a) ,  a  0,  which  characterizes  the  anatomical  structure  of  the  primate  and  human 
visual  systems.  4)  Showed  that  the  best  algorithm  for  fusing  multiple  space-variant 
fixations  of  the  same  scene  show,  under  certain  assumptions  of  pixel  distribution,  is 
indeed  optimal  in  a  least-squared-error  sense;  5)  Analyzed  the  characteristics  of  a 
synthetic  sensor  comparable,  with  respect  to  field  width  and  resolution,  to  the  prima 
visual  system;  6)  Showed  a  quantitative  measurement  of  the  macaque  ocular  dominance 
column  pattern,  based  on  measurement  of  local  power  spectral  densities  of  a  computer 
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June  30, 1991 


Dr.  Gen  Haddad  '* 

Life  Sciences  Directorate 

Department  of  the  Air  Force 

Air  Force  Office  of  Scientific  Research 

Bolling  Air  Force  Base,  D.C.  20332-6448 


Dear  Dr.  Haddad  , 


This  letter  constitutes  the  technical  report  for  the  period  June  1,  1990  ->  June  1,  1991  for  grant 
#AFOSR-88-0275.  The  contract  period  for  AFOSR-88-0275  is  scheduled  to  end  on  July  3 1,1991. 

Last  December,  we  received  a  letter  from  Dr.  John  Tangney  who  asked  contra ctees  to  attempt  to 
extend  existing  funding  for  as  long  as  possible,  due  to  possible  funding  restrictions  next  year  at  AFOSR,  so 
that  renewal  applications  could  be  pushed  into  next  year(Copy  enclosed). 

Since  we  have  made  substantial  progress  on  our  work  on  this  contract,  I  made  provisions  for  extend¬ 
ing  our  AFOSR  support  at  that  time  by  using  other  funding  available  to  support  part  of  the  personnel  on 
this  contract.  My  intention  is  to  finish  this  work  over  the  next  six  months,  and  to  write  a  renewal  applica¬ 
tion  in  the  fall,  as  per  Dr.  Tangney’s  suggestion  (in  December).  But  to  do  so  I  will  require  a  letter  of 
(unpaid)continuation  from  you  so  that  NYU  will  not  freeze  the  remaining  funding  on  this  contract.  I  dis¬ 
cussed  this  matter  with  Dr.  Tangney  on  the  telephone,  and  he  verbally  approved  it,  but  suggested  that  I 
write  to  you  describing  these  plans. 

If  possible,  it  would  be  helpful  if  you  wrote  a  note  approving  (in  principle)  this  unpaid  extension, 
since  otherwise  it  may  take  a  long  while  for  it  to  “grind”  through  both  the  DOD  and  NYU  beaurocracy, 
causing  a  delay  in  the  completion  of  the  contract.  Any  form  of  note  from  you  would  satisfy  the  NYU  end 
of  this,  (I  will  need  it  before  July  31,  to  avoid  any  problems  here).  I  am  very  appreciative  of  any  help  that 
you  can  provide  in  this  matter. 

I  will  now  briefly  outline  our  progress  during  the  past  year  by  outlining  our  publications  which  have 
been  submitted  and  which  have  appeared  in  print  in  this  period.  Copies  of  the  relevant  publications  are 
enclosed  with  this  report. 

1.  Computational  Neuroscience.  Edited  by  Eric  Schwartz.  MIT  Press,  1990. 

This  volume,  appearing  in  fall  1990,  presents  30  chapters  by  various  experts  in  computational  neu¬ 
roscience,  with  the  intention  of  providing  a  working  definition  of  this  term. 
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2.  Visualizing  and  understanding  patterns  of  brain  architecture.  Alan  Rojer  and  Eric  Schwartz,  to 
appear  in  Fall  1991,  NATO  ASR  series,  “  Biologically  Motivated  Robotic  Design”,  edited  by  P. 
Dario  and  G.  Sandini. 

We  illustrate  application  of  computer  science  to  neuroscience  at  three  levels:  measuring,  i  deling, 
and  understanding  the  computational  function  of  the  columnar  pattern  of  ocular  dominance  in  primate 
visual  cortex.  We  review  our  methods  for  the  quantitative  reconstruction  of  the  pattern  of  binocular  input  to 
the  visual  cortex  of  monkeys.  We  show  that  an  oriented  bandpass  filter,  applied  to  white  noise,  provides  a 
simple  parametric  characterization  of  the  observed  pattern.  We  suggest  a  computational  motivation  for  the 
columnar  architecture  as  a  “brain  data  structure’’  for  a  stereo  vision  algorithm  based  on  the  properties  of  a 
nonlinear  filter,  the  cepstrum.  This  work  dlustrates  some  of  the  algorithmic  difficulties  and  novel  research 
problems  encountered  when  computational  approaches  are  used  to  visualize  the  patterns  of  neural  architec¬ 
ture  of  the  primate  visual  system. 

3.  Computer  simulation  of  cortical  polymaps:  The  proto-column  algorithm.  Accepted  for  publication  in 
Ncui«i  ixetv*^.  ks(1991).  Pierre  Landau  and  Eric  Schwartz 

In  this  paper,  we  demonstrate, an  algorithm  for  modeling  polymap  architectures  of  the  cerebral  neo¬ 
cortex,  where  the  term  “polymap”  emphasizes  the  joint  occurrence  of  topographic  mapping  of  multiple 
sub-modalities,  interlaced  in  the  form  of  macroscopic  patches  (“columns”)  into  a  single  cortical  lamina 
Examples  include  the  ocular  dominance,  cytochrome  oxidase  puff/extra-puff,  and  orientation  column  struc¬ 
tures  of  primate  striate  cortex,  the  thick-thin-interstripe  columns  of  V-2,  and  the  direction  columns  of  MT. 
As  these  architectures  represent  the  merging  of  information  from  several  histological  and/or  featural  lam¬ 
inae,  we  use  the  term  “polymap”  to  describe  them.  The  justification  for  this  neologism  is  that  the  conven¬ 
tional  terms  “mapping”  and  “topographic  mapping”  are  not  sufficiently  expressive  of  the  complexity  and 
structure  of  neo-cortical  architecture,  and  their  use  often  leads  to  imprecision  or  confusion. 

The  structure  of  polymaps  raise  difficult  conceptual,  experimental,  and  computational  issues,  and  the 
demonstration  of  an  algorithm  for  simulating  them  is  of  fundamental  significance  to  the  understanding, 
description,  and  modeling  of  neo-cortical  functional  suchiteciurts.  Wc  iiave  found  it  necessary  to  introduce 
two  new  data  structures,  motivated  by  work  in  computational  geometry,  which  have  allowed  us  to  con¬ 
struct  a  detailed  algorithm  for  simulating  polymaps. 

4.  Design  considerations  for  a  space-variant  visual  sensor  with  complex  logarithmic  geometry.  Alan 
Rojer  and  Eric  Schwartz.  Proceedings  of  the  lO1*1  International  Conference  on  Pattern  Recognition 
(1990),  pp.  278  285. 

Human  vision  is  both  active  and  space-variant.  Recent  interest  in  exploiting  these  characteristics  in 
machine  vision  naturally  focuses  attention  on  the  design  parameters  of  a  space-variant  sensor.  We  consider, 
space-variant  sensor  design  based  on  the  conformal  mapping  of  the  half  disk,  w=log(z+fl),  with  real  a  >0, 
which  characterizes  die  anatomical  structure  of  the  primate  and  human  visual  systems.  There  are  ttuee 
relevant  parameters:  the  “circumferential  index”  k,  which  we  define  as  the  number  of  pixels  around  the 
periphery  of  the  sensor,  the  “visual  field  radius”  R  (of  the  half-disk  to  be  mapped),  and  the  “map  parame¬ 
ter”  a  from  above,  which  displaces  the  logarithm’s  singularity  at  the  origin  out  of  the  domain  of  the  map¬ 
ping.  We  show  that  the  log  sensor  requires  0(K2log(/?  la ))  pixels.  The  pixel  width  in  the  fovea  (foveal 
resolution)  L;w  is  proportional  to  a  Ik.  If  we  accept  a  fixed  circumferential  index  (constant  k),  the  space 
complexity  of  the  log  sensor  with  respect  to  foveal  resolution  goes  as  0(-logLfov).  By  contras*,  a 
uniform-resolution  sensor  has  a  spa'x  complexity  that  goes  as  OfLfo,-2)  Similarly,  when  the  space  com¬ 
plexity  ot  the  sensor  is  considered  with  respect  to  the  field  size  with  a  fixed  foveal  resolution,  we  find  that 
the  space  complexity  goes  as  0(\ogR ),  while  for  the  uniform-resolution  sensor,  the  space  resolution  goes 


as  0(R 2).  Using  this  analysis,  it  is  possible  to  directly  compare  the  space  complexity  of  different  sensor 
designs  in  the  complex  logarithmic  family.  In  particular,  we  can  obtain  rough  estimates  of  the  parameters 
necessary  to  duplicate  the  field  width/resolution  performance  of  the  human  visual  system. 

5.  Cat  and  monkey  cortical  column  patterns  modeled  by  band-pass  filtered  2D  white  noise.  Biological 
Cybernetics  62:331-391(1990).  Alan  Rojer  and  Eric  Schwartz 

A  simple  algorithm  based  on  bandpass-filtering  of  white  noise  images  provides  good  quality  com¬ 
puter  reconstruction  of  the  cat  and  monkey  ocular  dominance  and  orientation  column  patterns.  A  small 
number  of  parameters  control  the  frequency,  orientation,  “branchedness”,  and  “regularity”  of  the  column 
patterns.  An  oriented  (anisotropic)  bandpass  filter  followed  by  a  threshold  operation  models  the  macaque 
ocular  dominance  column  pattern  ’and  cat  orientation  column  system.  An  unoriented  (isotropic)  bandpass 
filter  models  the  cat  ocular  dominance  column  pattern  and  the  macaque  orientation  column  system.  The 
resemblance  of  computer  graphic  simulations  produced  by  this  algorithm,  and  histological  pattern  data,  is 
strong.  Since  this  algorithm  is  very  fast,  we  have  been  able  to  extensively  explore  its  parameter  space  in 
order  to  determine  filter  parameters  which  closely  match  the  structure  of  the  various  cortical  column  sys¬ 
tems.  In  particular,  we  have  applied  spectral  analysis  to  our  recent  computer  reconstruction  of  the  macaque 
ocular  dominance  column  system,  and  the  model  produced  by  the  present  algorithm  is  in  close  agreement 
with  this  detailed  data  analysis. 

6.  Fusion  of  multiple  fixations  with  a  space-variant  sensor:  conditional  optimality  of  maximum-resolution 
blending.  SPIE  Symposium  on  Advances  in  Intelligent  Systems,  Program  on  Intelligent  Robots  and 
Computer  Vision,  Intelligent  Robots  and  Computer  Vision  IX  (1990).  SPIE  1831-30,  pp.  1-11.  Alan 
Rojer  and  Eric  L.  Schwartz. 

A  unimodal  sensory  fusion  problem  posed  by  all  active  vision  systems  is  the  incorporation  of  multi¬ 
ple  fixations  (i.e.  views)  to  provide  a  coherent  and  stable  world  view.  The  degree  of  difficulty  of  this  prob¬ 
lem  is  greatly  increased  when  space-variant  active  vision  systems  are  considered.  In  such  systems,  in 
which  pixel  size  is  not  invariant  across  the  sensor,  the  same  regions  of  the  world  will  be  imaged  at  differing 
resolutions  in  successive  fixations.  Methods  for  fusing  multiple  fixations  with  a  space-variant  sensor  have 
received  little  analytic  or  computational  attention.  Mctivatation  derives  from  the  following  observations: 

•  High  performance  biological  systems  (caljnonkeyjiuman)  are  highly  space-variant. 

•  Construction  of  VLSI  based  space-variant  machine  vision  systems  is  currently  underway  in  several 

labs,  including  our  own. 

•  Active  vision  svstems  based  on  space-variant  sensors,  which  are  likely  to  dominate  some  applica¬ 
tion  areas  of  machine  vision  when  available,  require  a  solution  to  this  problem. 

In  previous  work  by  ourselves  and  others,  it  has  been  argued  that  the  best  algorithm  for  f»sing  multi¬ 
ple  space-variant  fixations  of  the  same  scene  is  to  simply  choose  the  maximum  resolution  information 
available  at  each  position  in  the  reconstructed  scene.  It  is  intuitively  compelling  that  this  will  yield  an 
optimal  recotiju notion,  since  it  uses  the  highest  resolution  (hence  most  accurate)  information  available  at 
each  position.  We  can  show,  under  certain  assumptions  of  p'xel  distribution,  that  this  algorithm  is  indeed 
optimal  in  a  least-squared-error  sense.  In  the  presence  of  noise,  however,  optimality  no  longer  holds;  a  cer¬ 
tain  degree  of  averaging  of  lower  resolution  information  is  required  to  obtain  optimal  reconstruction.  We 
also  plc"-.t  cmpirrai  der"cr.stationc  of  I  i;r.;.gcs  using  both  maximum-resolution  and  averaged 
blending  techniqes,  and  we  illustrate  the  effects  of  noise  on  the  quality  of  reconstruction. 


-4- 


7.  Biological  basis  for  space-variant  sensor  design  l:  Parameters  of  monkey  and  human  spatial  vision. 
SPIE  Symposium  on  Optics  Applications  (1990).  Eric  Schwartz  and  Alan  Rojer.  SPIE  1832-14,  pp. 
1-13. 

Biological  sensor  design  has  long  provided  inspiration  for  sensor  design  in  machine  vision.  However, 
relatively  little  attention  has  been  paid  to  the  actual  design  parameters  provided  by  biological  systems,  as 
opposed  to  the  general  nature  of  biological  vision  architectures.  In  the  present  paper,  we  will  provide  a 
review  of  cuiTent  knowledge  of  primate  spatial  vision  design  parameters  and  will  present  recent  experi¬ 
mental  and  modeling  work  from  our  lab  which  demonstrates  that  a  numerical  contormal  mapping,  which  is 
a  refinement  of  our  previous  complex  logarithmic  model,  provides  the  best  current  summary  of  this  feature 
of  the  primate  visual  system. 

8.  Biological  Basis  for  Space-Variant  Sensor  Design  II:  Implications  for  VLSI  Sensor  Design.  SPIE  Sym¬ 
posium  on  Advances  in  Intelligent  Systems  (1990).  SPIE  1386-06.  pp.  1-9.  Alan  Rojer  and  Eric 
Schwartz. 

We  analyze  the  characteristics  of  a  synthetic  sensor  comparable,  with  respect  to  field  width  and  reso¬ 
lution,  to  the  primate  visual  system.  We  estimate  that  150,000-300,000  pixels  are  sufficient  using  a  loga¬ 
rithmic  sensor  geometry,  and  demonstrate  that  this  calculation  is  consistent  with  known  characteristics  of 
biological  vision,  e.g.  the  number  of  fibers  in  the  optic  nerve.  To  obtain  the  field  width  and  resolution  of 
the  primate  eye  with  a  uniform  sensor  requires  between  KP-IO4  times  the  number  of  pixels  estimated  for 
the  comparable  log  sensor.  Another  interesting  observation  is  that  the  field  width  and  resolution  of  a  con¬ 
ventional  512x512  sensor  can  be  obtained  with  around  5000  pixels  using  the  log  geometry.  We  conclude 
with  consideration  of  the  prospects  for  achieving  human-like  performance  with  contemporary  VLSI  tech¬ 
nology  and  briefly  discuss  progress  on  space-variant  VLSI  sensor  design. 

9.  A  Quotient  Space  Hough  Transform  for  Space-Variant  Visual  Attention.  Submitted  to  Computer 
Vision,  Graphics  and  Image  Processing  (1991).  Alan  Rojer  and  Eric  L.  Schwartz 

We  consider  the  problem  of  “visual  attention”  in  the  context  of  space-variant  machine  vision:  Is 
there  a  general  theoretical  and  practical  formulation  for  the  “next-look”  problem  to  guide  a  space- variant 
sensor  to  a  rapid  choice  for  its  next  fixation  point?  This  topic  is  developed  in  the  context  of  Hough 
transform  methods,  by  the  addition  of  a  third  space  to  the  usual  feature  and  object  spaces  considered  in 
traditional  Hough  methods.  This  third  space  is  a  “behavioural,”  or  ’’motor”  space,  which  is  typically 
low-dimensional  with  respect  to  the  feature  and  object  spaces.  For  example,  the  motor  space  of  particular 
interest  for  us  is  the  two-dimensional  manifold  of  monocular  eye  positions.  By  “collapsing”  the  general¬ 
ized  Hough  transform  into  a  low  dimensional  motor  space,  we  show  that  it  is  possible  to  avoid  a  practical 
difficulty  in  applying  Hough  transform  methods,  which  is  the  exponential  dependence  of  the  accumulator 
array  on  the  dimensionality  of  the  object  space.  Beginning  with  a  simple  and  very  general  Bayesian 
scheme,  we  derive  in  stages  the  generalized  Hough  transform  as  a  special  case.  Since  “cttentional”  appli¬ 
cations,  by  their  nature,  require  only  partial  knowledge  about  objects,  computation  of  all  the  parameters 
characterizing  a  scene  object  is  superfluous  and  wasteful  of  computational  resources.  This  suggests  that  for 
an  attentional  application,  collapsing  the  (large)  object  space  onto  the  (small)  motor  space  provides  a  com¬ 
putationally  grounded  definition  of  the  term  “visual  attention”.  We  illustrate  these  ideas  with  an  example 
of  choosing  “fixation”  nomts  for  a  «p"C?  varipr*.  sensor  in  a  machine  vision  app'xauon  arl-nmc  read¬ 
ing  of  license  plates  of  moving  vehicles. 
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10.  A  Bayesian  Model  for  the  Hough  Transform.  Submitted  to  Pattern  Recognition  (1991).  Alan  Rojer 
and  Eric  L.  Schwartz. 

Although  the  Hough  transform  and  its  generalizations  have  been  used  for  many  years  in  computer 
vision  and  pattern  recognition,  its  conceptual  grounding  has  been  largely  heuristic.  We  demonstrate  here 
that  the  generalized  Hough  transform  may  be  viewed  as  a  special  case  of  Bayesian  inference.  Starting 
from  a  simple  and  very  general  Bayesian  scheme,  we  derive  in  stages  the  generalized  Hough  transform(of 
Ballard).  By  doing  so,  we  explicate  a  series  of  assumptions  concerning  featural  independence  and  equal  a 
priori  occurrence  of  objects  which  are  implicit  in  the  use  of  the  generalized  Hough  transform.  When  (if 
ever)  these  assumptions  are  satisfied,  Hough  transform  methods  are  optimal  (in  a  Bayesian  sense)  for 
object  recognition.  Consideration  of  the  assumptions  also  illuminates  conditions  under  which  performance 
degradation  of  the  Hough  approach  is  expected.  The  goal  of  this  paper  is  not  to  provide  yet  more  varia¬ 
tions  on  the  Hough  transform,  but  rather  to  clarify  the  implicit  assumptions  associated  with  this  methodol¬ 
ogy,  and  to  suggest  future  research  in  which  these  assumptions  can  verified  or  relaxed. 

11.  Measurement,  characterization  and  algorithmic  synthesis  of  the  macaque  ocular  dominance  column 
pattern.  Eric  L.  Schwartz  and  Alan  Rojer.  Invest.  Opthal.  and  Vis.  Sci.  31:1226  (1990). 

Columnar  patterns  in  the  visual  cortex  of  cats  and  monkeys  have  usually  been  measured  and 
described  by  qualitative  techniques.  The  goal  of  this  study  was  to  develop  quantitative  descriptive  methods 
for  columnar  patterns  in  visual  cortex.  In  the  present  work,  we  show  a  quantitative  measurement  of  the 
macaque  ocular  dominance  column  pattern,  based  on  measurement  of  local  power  spectral  densities  of  a 
computer  reconstruction  and  numerical  flattening  of  VI.  The  columnar  pattern  was  observed  following  sin¬ 
gle  eye  enucleation  and  cytochrome  oxidase  staining.  The  parameters  of  an  oriented  spatial  filter,  charac¬ 
terized  by  a  center  frequency,  a  bandwidth,  an  orientation,  and  an  aspect  ratio,  are  sufficient  to  locally 
describe  visual  cortical  columnar  patterns.  We  present  measurements  that  characterize  the  full  ocular  dom¬ 
inance  column  pattern  of  macaque  VI  in  this  way.  In  addition,  we  demonstrate  that  this  method  is  con¬ 
structive  as  well  as  descriptive.  By  applying  the  measured  filters  to  2D  white  noise,  followed  by  threshold¬ 
ing,  high  accuracy  models  of  the  original  column  patterns  can  be  algorithmically  constructed.  In  other 
words,  the  macaque  ocular  dominance  column  pattern  is  most  simply  described  in  terms  of  band-pass 
filtered  white-noise.  We  show  that  a  similar  algorithm  is  capable  of  algorithmically  reconstructing  the  cat 
(“patchy”)  ocular  dominance  column  pattern,  as  well  as  the  orientation  column  patterns  of  cat  and  mon¬ 
key.  Moreover,  we  demonstrate  that  many  of  the  previous  algorithms  that  have  been  suggested  to  account 
for  these  data  are  essentially  equivalent  to  our  bandpass  filtered  white  noise  model.  In  summary,  we 
present:  1.)  The  first  fully  quantitative  measurement  of  the  details  of  the  macaque  ocular  dominance 
column  pattern;  2.)  A  parametric  technique  for  describing  the  range  of  columnar  patterns  found  in  cat  and 
monkey  visual  cortex;  3.)  An  algorithm  for  synthesizing  columnar  patterns  which  is  much  faster  than  other 
method  with  which  we  are  familiar,  and  which  produces  synthetic  cortical  patterns  which  are  in  detailed 
qualitative  and  quantitative  agreement  with  the  measured  data. 

No  patent  applications  have  been  filed  during  this  period. 


Sincerely  yours, 


Eric  Setfwartz 
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